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TITLE OF THE INVENTION 

VIDEO SIGNAL CODING METHOD AND VIDEO SIGNAL ENCODER 
BACKGROUND OF THE INVENTION 
Field of the Invention 

This invention relates to a video signal coding method and a video signal 
encoder that can highly efficiently and effectively encode digital signals at the 
transmission side. More particularly, the present invention relates to a video signal 
coding method and a video signal encoder adapted to control a coding operation 
conducted at a variable bit rate on a single path when coding moving picture signals. 
Description of the Related Art 

Since digital video signals involve a large amount of data, the use of a high efficiency 
coding means that can encode a digital video signal at a high compression ratio is 
indispensable particularly when recording signals on a small recording medimn having 
a small storage capacity. High efficiency coding methods utilizing the correlation of 
video signals have been proposed as solutions for meeting the need. One of such 
solutions is the use of the MPEG system. The MPEG (Moving Picture Image Coding 
Experts Group) system was proposed as standard system as a result of the discussion 
in ISO-IEC/JTC1/SC2/WG11. It is a hybrid system reahzed by combining motion 
compensation predictive coding and discrete cosine transfonn (DCT). With the 
MPEG system, firstly the redundancy is eliminated on the time base by detennining 
the inter-fi-ame differences of video signals and subsequently on the space base by 
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means of discrete cosine transfonn to efficiently encode video signals. 

Generally, video signals do not remain invariable and the amount of infonnation 
of each picture changes with time. Thus, it is known that a high image quality can be 
realized by means of variable bit rate coding rather than by means of constant bit rate 
coding if a same amount of codes is used. 

For example, 2 -pass type variable bit rate coding is generally used for video 
signals to be recorded on a DVD-video. The 2-pass type involves two coding 
operations including one for detennining the amount of codes and one to be conducted 
by variably controlling the bit rate on the basis of the detennined amount of codes. 
While this technique provides the advantage of effectively exploiting the total amount 
of coding bits that can be used, it is accompanied by the disadvantage that a time 
period more than twice of the duration of the moving picture sequence is required for 
the processing operation so that this technique is not suited to real time processing 
operations. 

The specifications and the drawings of Japanese Patent Applications Laid-Open 
Nos. 7-311418 and 9-113141 disclose a 1-pass type variable bit rate coding method 
in an attempt for reducing the processing time. 

FIG. 1 of the accompanying drawings schematically illustrates the configuration 
of an encoder for moving pictures realized by applying the known 1-pass type variable 
bit rate coding method and FIG. 2 sows a flow chart of the processing operation using 
the 1-pass type variable bit rate coding method. 



• • • • 



Referring to FIG. 1 showing a block diagram of a known moving picture 
encoder, the input moving image signal applied to input terminal 200 is sent to coding 
difficulty level computation circuit 201 and delay circuit 203. Then, the output of the 
coding difficulty level computation circuit 201 is sent to amount of allocated bits 
computation circuit 202 that is adapted to compute the amount of allocated bits per 
unit time and the output of the amount of allocated bits computation circuit 202 is sent 
to moving image coding circuit 204. The moving image coding circuit 204 encodes 
the output signal of the delay circuit 203 as a function of the amount of allocated bits 
as output from the amount of allocated bits computation circuit 202 and outputs the 
coded signal as coded bit stream from terminal 205. 

Now, the operation of the moving picture encoder of FIG. 1 will be described 
by referring to the flow chart of FIG. 2. 

Referring to FIG. 2, in Step S40 1 , the moving picture signal fed to the tenninal 
200 is input to the coding difficulty level computation circuit 201 to detemune the 
coding difficulty level d of the input image per unit time, which unit time may typically 
be 0.5 seconds. The computing operation of the coding difficulty level computation 
circuit 20 1 is typically performed by encoding the input moving picture, fixing the 
quantization step, and computing the amount of generated codes in a predetennined 
period of time. 

Then, in Step S402, the amount of allocated bits computation circuit 202 
computationally detemiines the amount of allocated bits b relative to the coding 
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difficulty level d obtained by the coding difficulty level computation circuit 20 1 . For 
this operation, the relationship between the coding difficulty level d and the amount 
of allocated bits b observable when a reference moving image sequence is subjected 
to variable bit rate coding at a predetennined average bit rate is obtained in advance 
for each unit time. The amount of allocated bits per unit time for the reference moving 
image sequence is held to less than the storage capacity of the recording medium to 
be used for signal recording. FIG. 3 shows a graph that illustrates the relationship 
between the coding difficulty level d and the amount of allocated bits b. 

Referring to FIG. 3, the horizontal axis of the graph represents the coding 
difficulty level d and the vertical axis of the graph represents the probability of 
appearance h (d) of coding difficulty level d in the reference moving image sequence. 
The amount of allocated bits for any given coding difficulty level is computed on the 
basis of function b (d). This function is empirically obtained on a trial and error basis 
by conducting a number of experiments of encoding many moving image sequences 
(e.g., movies) at a predetennined average bit rate and evaluating the image quality of 
each sequence. In other words, it represents a generic relationship applicable to any 
moving image sequences that may be observable in this world. The specification and 
the drawings of Japanese Patent Application Laid-Open No. 7-311418 discloses a 
method of obtaining such a function. Thus, the amount of allocated bits computation 
circuit 202 detemiines the amount of allocated bits b for the coding difficulty level d 
per unit time of the image that is input through the terminal 200 on the basis of the 



relationship as illustrated in FIG. 3. 

Since the processing operation of the coding difficulty level computation circuit 
201 and the amount of allocated bits computation circuit 202 on the input image of a 
unit time length completes within the unit time, the delay circuit 203 is provided in the 
1-pass type moving picture encoder to delay the input of the image signal to the 
moving image coding circuit 204 by a unit time. 

Then, in Step S403, the moving image coding circuit 204 encodes the input 
moving image of each unit time so as to show the amount of allocated bits as given by 
the amount of allocated bits computation circuit 202 for the moving image. In other 
words, the moving image coding circuit 204 encodes the input moving image of each 
unit time, using the quantization step size that is defined on the basis of the amount of 
allocated bits. 

With such a 1-pass type method, it is possible to perform an operation of 
variable bit rate coding on the input video signal with an optimal amount of allocated 
bits that corresponds to the coding difficulty level of the signal substantially on a real 
time basis. 

Meanwhile, although the relationship of FIG. 3 is applicable to almost all 
moving image sequences, it is not to certain special sequences where the total amount 
of bits generated by the moving image coding circuit 204 exceeds the total amount of 
available bits so that the moving image sequence cannot be recorded on the intended 
recording medium. 



The conventional 2-pass type method cannot be used for encoding a moving 
image sequence having a predetemiined time length at a variable bit rate on a real time 
basis and recording it in a recording medium having a certain level of storage capacity. 
On the other hand, while the conventional 1-pass type method can encode almost all 
moving image sequences at a variable bit rate on a real time basis and store a sequence 
having a predetermined time length in a recording medium having a certain level of 
storage capacity, it is not applicable to certain special sequences where the total 
amount of coded bits exceeds the total amount of available bits so that the sequence 
having a predetennined time length cannot be recorded on a recording medium. 

In view of this problem, there has been proposed a technique of guaranteeing 
that a signal having a predetennined time length can be recorded on a recording 
medium having a certain level of storage capacity by controlling the amount of actually 
allocated bits in such a way that the sum of the bits generated when a signal having a 
time length good for recording on a recording medium is encoded is less than the 
amount of bits that can be used for recording the signal on the recording medium. 

More specifically, assuming that the input signal of each unit time is encoded 
with a predetermined amount of allocated bits b av, the sum of the amounts of 
allocated bits B av down to the current time and the sum of the amounts of actually 
generated coding bits B gen down to the current time are compared and if the value 
(B_av - Bgen) is positive, it is pennitted to give an amount of allocated bits more than 
b av for the guarantee. 



The predetennined amount of allocated bits b av per unit time is determined 
by the fonnula below; 

b_av = T_unit time x B_V/T_SEQ 
where B_V the amount of bits available to recording the moving image 

in the signal recording medium, 

T_SEQ : the time length of a moving image sequence that can be 
recorded in the signal recording medium, and 

T_unit time : unit time length. 

The input signal is pre-filtered on the basis of the information obtained when 
shifting the reference value of the amount of allocated coding bits to the actual value 
of the amount of allocated coding bits and then the obtained signal is coded to make 
the degradation of the signal due to the coding less conspicuous. More specifically, 
when the actual value of the amount of allocated coding bits is made smaller than the 
reference value of the amount of allocated coding bits, the degradation of the signal 
due to the coding less conspicuous by causing the input image to pass through a low 
pass filter. 

When the operation of coding a signal and recording the coded signal on a 
recording medium is divided into seginents of operation that are to be carried out 
sequentially, the above (B av - B gen) or a corresponding value is stored in the 
recording medium. Thus, when the signal is stored in a vacant storage area of the 
recording medium, the above (B av - B gen) or the corresponding value is read out 



from recording medium and the amount of bits allocated per unit time to the signal is 
computed on the basis of the read out value so that the storage capacity of the 
recording medium can be utilized effectively and efficiently. 

When the input signal is that of a moving image, the coding difficulty level is 
determined on the basis of the information on the image characteristics of the input 
image for every predetennined period of time and the signal is coded on the basis of 
the amount of allocated coding bits that reflects the visual characteristics of human 
being, using the infonnation on the image characteristics. 

FIG. 4 shows a block diagram of an encoder for moving pictures realized by 
applying a 1-pass type variable bit rate coding method that guarantees that a signal 
having a predetennined length on a recording medium having a certain level of storage 
capacity. 

Referring to FIG. 4, the moving image signal SI input to tenninal 101 is sent 
to a motion vector computation circuit 102. The motion vector computation circuit 
102 outputs infonnation on the motion vector (the motion vector and the prediction 
error) of the input moving image. The moving image signal and the motion vector 
infonnation are then input to a coding difficulty level computation circuit 103. The 
coding difficulty level computation circuit 103 computes the coding difficulty level d 
per unit time of the input moving image SI. The obtained coding difficulty level d is 
then input to the amount of allocated bits computation circuit 104 that computes the 
amount of allocated bits per unit time. More specifically, the amount of allocated bits 
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computation circuit 1 04 computes reference value b of the amount of bits allocated per 
unit time for coding the input image. The reference value b of the amount of allocated 
bits is then input to a controller 105. The controller 105 shifts the reference value b 
of the amount of allocated bits to the actual value b_x of the amount of allocated bits 
and outputs it. The controller 105 also outputs infonnation S4 on the processing 
operation of a pre-filter 107. A Delay circuit 106 delays the input of the input moving 
image signal S 1 having the length of the unit time to the pre-filter 1 07 by the unit time, 
or until the processing operation of the amount of allocated bits computation circuit 
104 and that of the controller 105 are completed. The input moving image signal S2 
delayed by the delay circuit 106 is processed by the pre-filter 107 according to the 
processing information S4 and the pre-filter 107 outputs the processed signal S3. The 
pre-filter 107 is adapted to change its filtering characteristics according to the 
processing information S4 so as to prevent the possible degradation of the input 
moving image signal at the time of coding by causing the signal to pass through a low 
pass filter when the input moving image is a complex one and requires a large amount 
of bits allocated to it per unit time. Moving image coding circuit 108 encodes the 
processed image signal S3 so as to make it show the amount of allocated bits b_x per 
unit time. The moving image coding circuit 108 then outputs coded bit stream S5 and 
the amount of generated bits b_gen per unit time, of which the coded bit stream S5 is 
output fi-om teraiinal 109. 

With such a 1 -path type method, the actual value of the amount of allocated bits 



can be controlled in such a way that input moving image can be coded at a variable bit 
rate by using an optimal amount of allocated bits according to the coding difficulty 
level of the signal on a real time bases and the total sum of the amounts of generated 
bits obtained when coding the signal having a time length that allows the signal to be 
recorded on the recording medium is less than the amount of bits available for 
recording the signal on the recording medium. 

However, the above described inethod for detennining the amount of allocated 
bits according to the coding difficulty level is accompanied by a problem that, when 
the input moving image signal shows a high coding difficulty level concentratively in 
the initial stages, (B av - B gen) would not show a positive value and hence it is not 
possible to improve the image quahty because a sufficient amount of bits cannot be 
allocated per unit time for the part of the image signal showing a high initial coding 
difficulty level if a latter part of the image signal shows a low coding difficulty level. 

Additionally, there is another problem that accompanies the above method. 
When (B av - B gen) becomes positive so that it is possible to allocate an amount of 
bits per unit time that is greater than the ordinary level and the input moving image 
signal continuously shows a high coding difficulty level so that an amount of bits per 
unit time that is greater than the ordinary level is actually allocated but subsequently 
(B av - B_gen) suddenly becomes negative on the way, the amount of allocated bits 
per unit time falls abruptly. 

On the other hand, when the input moving image signal continuously shows a 
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low coding difficulty level, there also arises a problem that the allocation of an amount 
of bits per unit time that is smaller than the ordinary level goes on to degrade the 
quality of the recorded image to a level lower than the image quality that is obtained 
when the image is coded at a fixed coding rate, although (B_av - B_gen) is positive 
and the amount of bits per that is allocated per unit time is greater than the ordinary 
level. 

Additionally, there is also a problem that, since the maximum amount of 
allocated bits and the minimum amount of allocated bits are not defined in actual 
operations, there can occur parts where the amount of allocated bits is extremely large 
and parts where the amount of allocated bits is extremely small after the coding 
operation and hence it is not possible to allocate bits to the input image in a well 
coordinated manner. 

Finally, although the image quality of an image can appear to be degraded in 
some parts to the visual sense of human being while it does not in some other parts if 
bits are allocated at a same rate, no satisfactory measures have been taken to such a 
problem. 

SUMMARY OF THE INVENTION 

In view of the above identified circumstances, it is therefore the object of the 
present invention to provide a video signal coding method and a video signal encoder 
that can allocate bits adequately and follow any changes in the coding difficulty level 
so that the input video signal can be coded effectively and efficiently even the input 
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video signal concentratively shows a high coding difficulty level in the initial stages, 
when it is possible to allocate bits at a rate higher than the ordinary rate and the input 
video signal continuously shows a high coding difficulty level and when the input 
video signal continuously shows a low coding difficulty level. 

In an aspect of the invention, the above object is achieved by providing a video 
signal coding method comprising: 

a step of determining the coding difficulty level d of an input video signal for 
each unit time; 

a step of determining a reference value for allocating coding bits on the basis 
of function b (d) for the amount of coding bits b allocated for each unit time and 
related in advance to the coding difficulty level d of said input video signal for each 
unit time; 

a step of determining the actual amount of allocated coding bits b_x on the basis 
of the reference value; and 

a step of generating coded data by coding the input video signal for each unit 
time on the basis of said actual amount of allocated coding bits b_x. 

When obtaining the actual amount of allocated coding bits on the basis of the 
above reference value in order to shift the reference value of the amount of allocated 
coding bits to the actual value of the amount of allocated coding bits, part of the sum 
of the amounts of allocated coding bits per unit time is stored as virtual buffer for a 
certain period of time in advance and the actual reference value of the amount of 
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allocated coding bits is detennined by dividing the sum of the amounts of allocated 
coding bits per unit time less the part stored as virtual buffer by said certain period of 
time. As a result, it is possible to allocate bits at a rate higher than the reference value 
of the amount of allocated bits per unit time if the input moving image signal 
continually shows a high coding difficulty level in the initial states of the coding 
operation that lasts for said certain period of time. 

More specifically, when shifting the reference value of the amount of allocated 
coding bits to the actual value of the amount of allocated coding bits, part of the sum 
B av of the amounts of allocated bits b_av per unit time for a certain period of time 
T_vbr, or 

B av = b av >< T vbr, 
is stored as virtual buffer V vbr in advance and the actual reference value of the 
amount of allocated coding bits b real is obtained by 

b real = (B av - V vbr) / T vbr 
so that an amount of allocated bits not smaller than b real is given as long as V vbr 
> 0 but an amount smaller than b real is given otherwise. 

As a result, when the coding operation is controlled on the basis of a variable 
bit rate, it can be guaranteed that the sum of the generated bits for said certain period 
of time T vbr is less than B av. 

When correlating the coding difficulty level of the input image with the amount 
of allocated bits per unit time, the reference value is determined by taking the 
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relationship of the coding difficulty level, the amount of allocated coding bits and the 
amount of actually generated bits of a preceding unit time and also the relationship of 
the coding difficulty level of the preceding unit time and that of the current unit time. 
As a result, the amount of allocated bits is detemiined by taking the coding difficulty 
level and the amount of allocated bits of the preceding time unit into consideration 
when the input video signal continuously shows a high coding difficulty level or a low 
coding difficulty level for some time so that a situation where the amount of allocated 
bits abruptly falls on the way. Addifionally, if possible, a large amount of bits may be 
allocated when the coding difficulty level of the input video signal is low. 

Still additionally, according to the invention, when bits are allocated per unit 
time by an amount greater than or smaller than the reference value, it is possible 
allocate bits in a well coordinated manner by defining in advance an upper limit and 
a lower limit for the amount of allocated bits per unit time as a fiinction of the 
reference value. Then, a situation where an extremely large or small amount of bits 
are allocated locally can be effectively avoided. 

Furthen-nore, when an amount of bits less than the reference value is allocated 
because the coding difficulty level is low, a lower limit may be defined as a fiinction 
of the ratio of the parts where degradation due to coding is conspicuous in order to 
prevent a coding operation that remarkably degrade the image quality fi-om occurring 
by taking the visual characteristics of the input video signal into consideration. 

As pointed out above, according to the invention, when a reference value (for 
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the amount of allocated coding bits) is detemiined on the basis of the amount of 
allocated codingbits obtained from the relationship with the coding difficulty level and 
the amount of actually allocated coding bits is detemiined on the basis of the 
reference, part of the sum of the amounts of allocated coding bits per unit time is 
stored as virtual buffer for a certain period of time in advance and the actual reference 
value of the amount of allocated coding bits is detennined by dividing the sum of the 
amounts of allocated coding bits per unit time less the part stored as virtual buffer by 
said certain period of time. As a result, it is possible to allocate bits at a rate higher 
than the reference value of the amount of allocated bits per unit time if the input 
moving image signal continually shows a high coding difficulty level in the initial 
states of the coding operation that lasts for said certain period of time. 

When correlating the coding difficulty level of the input image with the amount 
of allocated bits per unit time, the reference value is detennined by taking the 
relationship of the coding difficulty level, the amount of allocated coding bits and the 
amount of actually generated bits of a preceding unit time and also the relationship of 
the coding difficulty level of the preceding unit time and that of the current unit time 
into consideration. As a result, the amount of allocated bits is detennined by taking 
the coding difficulty level and the amount of allocated bits of the preceding time unit 
into consideration when the input video signal continuously shows a high coding 
difficulty level or a low coding difficulty level for some time so that a situation where 
the amount of allocated bits abruptly falls on the way. Additionally, if possible, a large 
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amount of bits may be allocated when the coding difficulty level of the input video 
signal is low. 

Furthermore, according to the invention, a lower limit may be defined as a 
function of the ratio of the parts where degradation due to coding is conspicuous in 
order to prevent a coding operation that remarkably degrade the image quality from 
occurring by taking the visual characteristics of the input video signal into 
consideration. 

Thus, according to the invention, it is possible to guarantee that an input video 
signal having a predetennined time length is stored in a recording medium having a 
certain storage capacity to effectively utilize the storage capacity of the recording 
medium, while controlling the coding operation adaptively using a variable bit rate as 
a function of the coding difficulty level of the input video signal. BRIEF 
DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic block diagram of a known moving image encoder realized 
by applying a conventional basic 1-pass type variable bit rate coding method; 

FIG. 2 is a flow chart of the conventional 1-pass type variable bit rate coding 
method; 

FIG. 3 is a graph of the function b (d) showing the relationship between the 
probability of appearance h (d) per unit time length of coding difficulty level d in the 
reference moving image sequence and the amount of allocated bits for a predetermined 
average bit rate b. 
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FIG. 4 is a schematic block diagram of a known moving image encoder realized 
by applying a conventional evolutional 1 -pass type variable bit rate coding method; 

FIG. 5 is a schematic block diagram of an embodiment of moving image 
encoder realized by applying a video signal coding method according to the invention; 

FIG. 6 is a flow^ chart illustrating the algorithin of the amount of allocated bits 
per unit time computation circuit and the controller 24 of FIG. 5; 

FIG. 7 is a flow chart of the operation of correcting the coding difficulty level 
in Step S56 of FIG, 6; 

FIG. 8 is a flow chart of the operation of visually correcting the amount of 
allocated bits in Step S58 of FIG. 6; 

FIG. 9 is a flow chart of the operation of correcting the amount of allocated bits 
in Step S59 of FIG. 6; 

FIG. 10 is a flow chart of the operation of correcting the ainount of allocated 
bits for the absolute coding difficulty level in Step S60 of FIG. 6; 

FIG. 1 1 is a flow chart of the operation of processing a scene change in Step 
S62 of FIG. 6; 

FIG. 12 is a flow chart of the operation of processing a virtual buffer in Step 
S63ofFIG. 6; 

FIG. 13 is a schematic illustration of the concept of virtual buffer; and 
FIG. 1 4 is schematic block diagram of a signal recording/reproduction apparatus 
to which an embodiment of encoder according to the invention is applicable. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Now, a video signal coding method and a video signal encoder according to the 
invention will be described by referring to the views of the accompanying drawing that 
illustrate preferred embodiments of the invention particularly in tenns of encoding a 
moving image. 

FIG. 5 is a schematic block diagram of an embodiment of moving image 
encoder realized by applying a video signal coding method according to the invention. 

Referring to FIG. 5, moving image signal S 1 input to a tenninal 3 1 is sent to a 
motion vector computation circuit 32, a coding difficulty level computation circuit 33, 
a visual characteristics measurement circuit 35 and a delay circuit 36. 

The visual characteristics measurement circuit 35 measures visual 
characteristics of the input moving image signal S 1 and sends the obtained information 
on the visual characteristics R_visual to amount of allocated bits computation 
circuit/controller 34. The motion vector computation circuit 32 outputs the infomiation 
obtained for the motion vector (the motion vector and the prediction error) of the input 
moving image. The infonnation on the motion vector is sent to the coding difficulty 
level computation circuit 33 along with the moving image signal SI. 

The coding difficulty level computation circuit 33 computes the coding 
difficulty level d for each unit time from the infonnation on the motion vector sent 
from the motion vector computation circuit 32, the information on the amount of 
generated bits b gen fed from moving image coding circuit 38 which will be described 
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hereinafter, Q scale q_scale and the deviation of the brightness value activity. The 
coding difficulty level d is then input to the amount of allocated bits computation 
circuit/controller 34. While I GOP (gioup of pictures) according to MPEG may 
typically be used for the unit time for the purpose of the invention, the present 
invention is by no means limited thereto. 

The amount of allocated bits computation circuit/controller 34 computes the 
amount of bits to be allocated b_x for each unit time when coding the input moving 
image fi-om the coding difficulty level d input to it, the average amount of allocated 
bits b av input to it fi-om the terminal 30 and the visual characteristics R visual fed 
fi-om the visual characteristics measurement circuit 35. The obtained amount of 
allocated bits b_x is then output to moving image coding circuit 38. The amount of 
allocated bits computation circuit/controller 34 also outputs processing information 
S4 to pre-filter 37. 

The delay circuit 36 delays the input of the input moving image signal S 1 to the 
pre-filter 37 by a unit time until theprocessing operations of the coding difficulty level 
computation circuit 33 and the amount of allocated bits computation circuit/controller 
34 are completed. The moving image signal S2 delayed by the delay circuit 36 is then 
processed by the pre-filter 37 according to the processing infonnation S4, which then 
outputs processing infonnation S3 to the moving image coding circuit 38. 

The moving image coding circuit 38 encodes the processed video signal S3 for 
each unit time in such a way that it shows the above amount of allocated bits b_x. The 
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moving image coding circuit 38 outputs the coded bit stream S5, the amount of 
generated bits b_gen for each unit time, the deviation of the brightness value activity 
and the allocated Q scale q_scale. The coded bit stream S5 is output from the tenninal 
39. 

With the above arrangement, the coding difficulty level computation circuit 33 
and the amount of allocated bits computation circuit/controller 34 detennines a 
reference value (e.g., a reference value for the amount of allocated coding bits) on the 
basis of the amount of allocated coding bits obtained from the relationship with the 
coding difficulty level and stores in advance part of the sum of the amounts of 
allocated bits per unit time for a certain period of time as virtual buffer so that the 
actual reference value of the amount of allocated coding bits is detennined by dividing 
the sum of the amounts of allocated coding bits per unit time less the part stored as 
virtual buffer by said certain period of time. 

More specifically, when detennining the reference value for the amount of 
allocated coding bits relative to the coding difficulty level d of each unit time and 
shifting the reference value of the amount of allocated coding bits to the actual value 
of the amount of allocated coding bits b_s , part of the sum B av of the amounts of 
allocated bits b av per unit tiine for a certain period of time T vbr, or 

B av = b av x T vbr, 
is stored as virtual buffer V vbr in advance and the actual reference value of the 
amount of allocated coding bits b real is obtained by 
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b real = (B av - V vbr) / T vbr 
so that an amount of allocated bits not smaller than b real is given as long as V vbr 
> 0 but an amount smaller than boreal is given otherwise. While the period of time 
T_vbr may be about 1 minute, the present invention is by no means limited thereto. 

The reference value is detennined by taking the relationship of the coding 
difficulty level, the amount of allocated coding bits and the amount of actually 
generated bits of a preceding unit time and also the relationship of the coding difficulty 
level of the preceding unit time and that of the current unit time into consideration. 

It is determined in advance that the amount of the upper limit corresponding to 
the amount of allocated bits b av per unit time is given when allocating an amount of 
bits exceeding the above breal and the amount of the lower limit corresponding to the 
amount of allocated bits bav per unit time is given when allocating an amount of bits 
short of the above b real. 

Additionally, the video signal of images whose degradation due to coding can 
become conspicuous is provided with a lower limit for the amount of allocated bits 
corresponding to the possible degree of degradation, taking the visual characteristics 
of human being into consideration. 

If the difference between the sum of the amounts of actually generated bits 
B gen in the period of time and the sum of the amounts of available bits B av in the 
period of time (B av - B gen) is positive when the coding operation in said period of 
time Tvbr is over, the difference is carried over and added to the sum of the amounts 
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of available bits in the next period of time. If the sum of the amounts of available bits 
exceeds R total times of the initial sum B av as a result of carrying over the 
difference, the reference value of the actually allocated bits per unit time boreal is 
raised as a function of the ratio. 

Now, the algorithin for determining the amount of allocated bits of the coding 
difficulty level computation circuit 33 and the amount of allocated bits computation 
circuit/controller 34 of FIG. 5 will be described by referring to the flow charts of FIGS. 
6 through 12. 

Referring firstly to FIG. 6, the processing operation for detennining the amount 
of allocated bits starts in Step S50 and then the coding difficulty level computation 
circuit 33 inputs the moving image in Step S51 and initializes the reference value of 
coding difficulty level d_Avg by using initial value d Default in Step S52. 

Then, in Step S53, the reference bit rate b real when the variable bit rate is 
controlled is determined from the average amount of allocated bits b av per unit time 
input from the tenninal 30 of FIG. 5 by using fonnula 

b real = b av x R vbr [Mbps]. 
Note that the ratio of the reference bit rate when the variable bit rate is controlled, or 
Rvbr is defined by 0 < Rvbr < 1 . For instance, a value of Rvbr ^ 0.9 may be used. 
The time period of 1 GOP (group of pictures) according to MPEG may typically used 
for the unit time. 

In Step S54, virtual buffer V vbr for controlling the variable bit rate is secured 
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for the time when the control of the variable bit rate for the period of time T vbr is 
started by using the formula below. If the temporally preceding coding operation has 
any residue in V_vbr, the residue is carried over to the succeeding period of time 
Tvbr. 

V_vbr ^ V_vbr + ( 1 - R_vbr) x T_vbr x b_av [Mbit] 
By utilizing the virtual buffer V_vbr, an amount of infonnation greater than b_real is 
allocated to an image showing a relatively high coding difficulty level, whereas an 
amount of information smaller than b real is allocated to an image showing a 
relatively low coding difficulty level. 

Now, the concept of virtual buffer will be described by referring to FIG. 13. In 
FIG. 13, part of the sum B_av (= b_av x T vbr) of the amounts of allocated bits per 
unit time obtained for the time period T vbr is stored in advance as virtual buffer 
Vvbr. If the amount of bits per unit time of the virtual buffer is vvbr, Vvbr = vvbr 
X Tvbr. Additionally, the reference value b_real of the amount of actually allocated 
bits per unit time is expressed by b_real = (B av - V vbr) / T vbr. For the time period 
T vbr, a value of T_vbr = 1 [min] may typically be used. 

In Step S55, the coding difficulty level d of the unit time for which the image 
signal is being coded currently and the visual characteristics R visual are input to the 
amount of allocated bits computation circuit/controller 34 for each unit time. 

The parameters that can be used for estimating the coding difficulty level d per 
unit time may include the prediction error of the unit time preceding the above unit 
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time ME_error, the motion vector MV, the index indicating the deviation of the 
brightness value activity, the amount of generated infonnation bgen and the Q scale 
q_scale. 

The coding difficulty level computation circuit 33 estimates the coding 
difficulty level d, using the fonnula below. 

d = ME_error x W_ME + MV x W_MV + activity x W_AC + b_gen x 
W_GEN + q_scale x W_Q, 
where 

W ME weight to the motion prediction error, 

W MV weight to the motion vector, 

W_AC : weight to the deviation of the brightness value 

W GEN : weight to the amount of generated bits and 

W_Q : weight to the Q scale. 

In Step S56, the input coding difficulty level d is corrected. This is a processing 
operation for limiting the coding difficulty level d by means of the upper limit value 
d_Max and the lower limit value d_Min. FIG. 7 is a flow chart illustrating the 
operation in detail. 

Referring to FIG. 7, the operation of correcting the coding difficulty level is 
started in Step S70 and, if it is found in Step 871 that the estimated coding difficulty 
level exceeds the upper limit value d_Max of coding difficulty level, the coding 
difficulty level of the unit time is detennined to be equal to d_Max in Step S73. If, on 
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the other hand, it is found in Step S72 that the estimated coding difficulty level is 
smaller than the lower limit value d Min of coding difficulty level, the coding 
difficulty level of the unit time is determined to be equal to d Min in Step S74. In 
other words, 

if (d > d_Max), d = d_Max; 

if(d<d_Min), d-d_Min; 
After Steps S72, S73 and S74, the operation of correcting the coding difficulty level 
ends in Step S75. 

Now, returning to FIG. 6, the amount of allocated bits b_x is determined in Step 
S57 by using the reference value of amount of allocated bits b_real, the coding 
difficulty level d and the reference value of coding difficulty level d_Avg. In other 
words, 

b_x = b_real x d / d_Avg [Mbps] 

In Step S58, the lower limit of amount of allocated bits b_Min is regulated or 
corrected according to the visual characteristic value R visual because the perceived 
degradation of image quality due to the coding operation can vary depending on the 
visual characteristics of the input image. FIG. 8 shows a flow chart of the operation 
ofStepSSS. 

Referring to FIG. 8, the operation of correcting the visual characteristic value 
starts in Step S80 and, if it is found in Step S81 that the visual characteristic value 
Rvisual of the input image is greater than a predetermined value Th_visual, the lower 
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limit b_Min of amount of allocated bits is corrected by using the fon-nula below in Step 
S82. That is, 

if (R visual > Th_visual) 

b_Min = b_Min x (1 - Th_visual + R_visual); 

If it is found in Step S83 that the corrected lower limit value b_Min of amount 
of allocated bits exceeds the upper limit value b_Max of amount of allocated bits, the 
lower limit value b_Min is made to be equal to upper limit value b_Max of amount of 
allocated bits in Step S84. In other words, 

if (b_Min > b_Max), b_Min = b_Max; 
The visual characteristic value as used herein may refer to the ratio of the skin color 
or the intermediary tones in the input image and be expressed in tenns of 0 < 
Th_visual < 1 and 0 < R visual < 1. Since Th visual represents a constant value, it 
may typically be made equal to 0.5. After Steps S81 and S84, the operation of 
correcting the visual characteristic value ends in Step S85. 

Returning again to FIG. 6, in Step S59, the amount of allocated bits per unit 
time detennined in Step S57 is corrected by referring to the upper limit value b_Max 
and the lower limit value b_Min of amount of allocated bits that are predetermined. 
FIG. 9 is a detailed flow chart of the operation of correcting the amount of allocated 
bits. 

Referring to FIG. 9, the operation of correcting the amount of allocated bits 
starts in Step S90 and, if it is found in Step S91 that the amount of allocated bits b_x 

26 



exceeds the upper limit value b_Max of amount of allocated bits, the amount of 
allocated bits is made equal to b_Max in Step S93. If, on the other hand, it is found 
in Step S92 that the amount of allocated bits b_x is lower than the lower limit value 
b_Min of amount of allocated bits, the amount of allocated bits is made equal to 
b_Min in Step S94. In other words, 

if (b_x > b_Max), b_x = b_Max; 

if (b_x < b_Min), b_x = b_Min; 
The following values may typically be used respectively for the upper limit value 
b_Max of amount of allocated bits and the lower limit value of b_]Vlin of amount of 
allocated bits; 

b_Max = 2.0 b_real, and bJVIin = 0.5 b real; 
After Steps S92, S93 and S94, the operation of correcting the amount of allocated bits 
ends in Step S95. 

Now back to FIG. 6, the amount of allocated bits is corrected in Step S60 on the 
basis of the predetermined absolute rating for the coding difficulty level. FIG. 10 is 
a detailed flow chart of the operation of correcting the absolute amount of bits. 

Referring to FIG. 10, the operation of correcting the absolute amount of bits 
starts in Step SI 00 and then, if it is determined in Step SlOl that the estimated coding 
difficulty level d exceeds the upper limit of absolute value of coding difficulty level 
d High and the amount of allocated bits b_x falls below the lower limit of absolute 
value of amount of allocated bits b_ Low, the amount of allocated bits is made equal 
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to b_Low in Step SI 03. If, on the other hand, it is detennined in Step SI 02 that the 
estimated coding difficulty level d falls under the lower hmit of absolute value of 
coding difficulty level d_Lowh and the amount of allocated bits b_x exceeds the upper 
limit of absolute value of amount of allocated bits b_ High, the amount of allocated 
bits is made equal to b High in Step SI 04. In other words, 

if (d > d_High b_ x < b_Low) 

b_x = b_Low; 

if (d < d_Low && b_ X > b_High) 
b_x — bHigh; 

The following values may typically be used respectively for the lower limit of absolute 
value of amount of allocated bits b_Low and the upper limit of absolute value of 
amount of allocated bits b High; 

b_Low = 1 .2 b_real, b_High = 0.8 b_real; 
After Steps S 102, S 103 and S 1 04, the operation of correcting the amount of allocated 
bits ends in Step S105. 

Referring back to FIG. 6 at step S61, the reference value of coding difficulty 
level d_Avg is updated according to the reaction speed Reactor, using the coding 
difficulty level d of the current unit time. In other words, 

d_Avg = (d_Avg X (1 - Reactor) + d x Reactor) 
where the reaction speed Reactor is 0 < Reactor < 1 and may typically be 0.5. 

In Step S62, if there is a scene change, a processing operation for the scene 
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change is conducted. FIG. 11 illustrates a detailed flow chart of the operation for 
processing a scene change. 

Referring to FIG. 1 1, the operation of processing a scene change starts in Step 
S 1 1 0 and, if a scene change is detected in Step S 1 1 1 , the reference value of coding 
difficulty level d_Avg is initialized by initial value d_Default. If it is found in Step 
S 1 13 that the amount of infonnation necessary for the virtual buffer V vbr is left, the 
allocated bit rate is corrected in Step SI 14 according to the ratio of the allocated bit 
rate R_SC immediately after the scene change. This processing operation can be 
expressed as follows; 

if (scene_change) { 
d_Avg = d_Default; 
if (V_vbr> (R_SC -1) X b_vbr x 0.5) { 
b_x = b_av X R_SC; 

- } ■ 
} 

where the ratio of the allocated bit rate RSC is 1 < RSC and may typically be RSC 
= 1.5. After Steps S 1 1 1 , S 1 13, S 1 14, the operation of processing the scene change 
ends in Step SI 15. 

Then, in Step S63 of FIG. 6, a processing operation relating to the virtual buffer 
V_vbr is carried out. FIG. 12 illustrates a detailed flow chart of the operation of 
processing the virtual buffer. 
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Referring to FIG. 12, the operation of processing the virtual buffer starts in Step 
SI 20 and, if it is found in Step S121 that the residue of virtual buffer V_vbr is less 
than 0 and the amount of allocated bits b_x is less than the reference value of amount 
of allocated bits b real, the operation proceeds to Step SI 25, where the amount of 
allocated bits b_x is made equal to the reference value of amount of allocated bits 
b_real. In other words, 

if ( V_vbr < 0 && b_x > b_real) 
b_x = breal; 

Then, if it is found in Step S122 that the virtual buffer V vbr exceeds the 
predetermined upper limit value of virtual buffer V_Max, the amount of allocated bits 
b_x is corrected in Step SI 23 by using the following formula. 

if(V_vbr> V_Max) { 

b_x = V vbr / V Max x b_x; 

} 

The upper limit value of virtual buffer V Max may typically be expressed by 
V_Max = 2.0 V_vbr; 

Then, in Step S 124, the difference between the amount of allocated bits b_x and 
the reference value of amount of allocated bits b real is subtracted from the residue 
of virtual buffer V_vbr to update the residue of virtual buffer V vbr. After Steps 
S125, S124, the operation of processing the virtual buffer ends in Step S126. 

Referring again back to FIG. 6, if it is found in Step S64 that the unit time 
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(GOP) for which the processing operation is being conducted is the last unit time of 
the period of time Tvbr, the operation proceeds to Step S63. Otherwise, the operation 
returns to Step S55, where the processing operation for the next unit time starts. If it 
is found in Step S65 that the unit time is the last one for the processing sequence, the 
operation proceeds to Step S66, where the processing operation ends. Otherwise, the 
operation returns to Step S55, where the virtual buffer V_vbr is supplemented and the 
processing operation for the next period of time T_vbr starts. 

Now, a signal recording/reproduction apparatus to which this embodiment can 
be apphed will be described in detail by referring to FIG. 14. The apparatus of FIG. 
14 is adapted to use a hard disk as recording medium. 

Referring to FIG. 14, the television broadcast signal received typically by means 
of an antenna is transmitted to a tuner 1 , which may be a terrestrial wave tuner, by way 
of input tenninal 101. The video output and the audio output of the tuner 1 are then 
fed to an input switch circuit 2. Similarly, the composite video signal and the audio 
signal from an external source are also fed to the input switch circuit 2 via the input 
terminal 102 and input terminal 103 respectively. The input switch circuit 2 select the 
desired signals according to the instmction from system controller 16. The composite 
video signal is output to Y/C separation circuit 3, while the audio signal is output to 
audio A/D converter 10. 

The system controller 16 controls all the component blocks of the apparatus, 
which will be described hereinafter. Additionally, if necessary, the system controller 
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16 accesses and controls ROM (read only memory) 17 and/or RAM (random access 
memory) 18. 

The Y/C separation circuit 3 separates the Y (luminance) signal and the C 
(chroma) signal out of the composite video signal it receives, generates an elementary 
stream (ES) of the image and feeds them to input switch 4. The input switch 4 selects 
either the signal from external S video input tenninal 104 or the output of the Y/C 
separation circuit 3 and feeds the selected signal to NTSC decoder circuit 5. 

The video signal input to the NTSC decoder 5 is subjected to A/D conversion, 
chroma encoding and other operations and converted into a digital component video 
signal (to be referred to as video data hereinafter), which is then fed to pre-processing 
(pre-video signal processing) circuit 7. Additionally, the NTSC decoder 5 supplies the 
clock generated by referring to the horizontal synchronizing signal of the input video 
signal and the horizontal synchronizing signal, the vertical synchronizing signal and 
the field discriminant signal obtained by sync separation to sync control circuit 6. 

The sync control circuit 6 generates a clock and a sync signal adapted to the 
timings required for the blocks as will be described hereinafter and supplies them to 
each of the blocks. 

The pre-processing (pre-video signal processing) circuit 7 performs various 
video signal processing operations such as pre-filtering and noise reduction on the 
input video data and supplies the processed signal typically to MPEG (moving picture 
experts group) encoder 8 and post-processing (post- video signal processing) circuit 20. 
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The MPEG encoder 8 perfomis various coding operations such as block 
detection (discrete cosine transfonn) on the video data from the pre-processing circuit 
7, generates an elementary stream (ES) of the image and supplies the coded signal to 
multiplexer/demultiplexer 9. While the MPEG compression system is used in this 
embodiment, some other compression system may alternatively be used for the 
purpose of the invention. Furthennore, the signal may not necessarily be compressed. 

On the other hand, the audio signal selected by the input switch circuit 2 is 
converted into a digital audio signal by the audio A/D converter 10 and then fed to 
audio encoder 1 1 which may typically be an MPEG audio encoder. The signal is then 
compressed by the MPEG audio encoder according to the MPEG fonnat and 
subsequently an ES is generated so that the signal and the ES are supplied to the 
multiplexer/demultiplexer 9 as in the case of video signal. As for audio signals, while 
the MPEG compression system is used in this embodiment, some other compression 
system may alternatively be used for the purpose of the invention. Furthennore, the 
signal may not necessarily be compressed. 

For a recording operation, the multiplexer/demultiplexer 9 multiplexes the video 
ES (elementary streain), the audio ES and the various control signals. For a 
reproducing operation, it separates the transport stream (TS). The 
multiplexer/demultiplexer multiplexes the MPEG video ES (elementary stream), the 
MPEG audio ES and the various control signals and typically generates a TS of the 
MPEG system. Then, it feeds them to buffer control circuit 14. 
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The buffer control circuit 14 controls the operation of intennittently 
transmitting the TS that is input continuously to downstream hard disk drive (HDD) 
1 5. For example, since the HDD (hard disk drive) 1 5 is not available for writing the 
TS signal when it is seeking data, the TS signal is temporarily stored in a buffer. On 
the other hand, when the HDD 15 is available for writing the TS signal, the latter is 
written at a rate higher than the rate at which it is input so that the input TS signal that 
is input continuously can be recorded without any omission. 

The HDD 1 5 records the TS signal at predetennined addresses under the control 
of the system controller 16. 

While IDE (intelligent drive electronics) is used as protocol for the buffer 
control circuit 14 and the HDD 15 in this embodiment, the present invention is by no 
means limited thereto. Similarly, while this embodiment is described above by 
referring to an HDD as recording medium, it may be replaced by an optical disk, a 
magneto-optical disk or a solid memory without any problem. 

Now, the signal reproducing operation of the embodiment will be described 

below. 

The HDD 15 seeks predetemiined addresses to read the TS (transport stream) 
signal and supplies it to the buffer control circuit 14 under the control of the system 
controller 16. The buffer control circuit 14 controls the buffer so as to turn the signal 
that is input intennittently into a continuous signal and feeds the TS signal to 
multiplexer/demultiplexer 9. The multiplexer/demultiplexer 9 extracts the PES 
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(packetized elementary stream) out of the time stamp and feeds it to AV (audio/video) 
decoder 19, which may typically be an MPEG AV decoder. 

The MPEG AV decoder 1 9 separates the video ES and the audio ES of the input 
PES and feeds them respectively to the video MPEG decoder and the audio MPEG 
decoder for decoding. As a result of this processing operation, video data and the 
audio data of base band are obtained and fed to post-processing (post-video signal 
processing) circuit 20 and switch 23. 

The post-processing circuit perforais operations such as selecting the video data 
from the pre-processing (pre-video signal processing) circuit 7, synthetically 
combining signals and/or filtering signals and supplies the obtained video data to OSD 
(on screen display) circuit 21. 

The OSD circuit 2 1 generates graphics to be displayed on the display screen and 
perforins other operations such as superimposing them on video data or displaying 
them in part of the screen. The obtained video data are then fed to the NTSG encoder 
22. 

The NTSC encoder 22 converts the input video data (component digital signal) 
into a YC (luminance, chroma signal), which is then subjected to D/A conversion to 
produce an analog composite video signal and an S video signal, which composite 
video signal and S video signal are then taken out externally by way of output temiinal 
106 and output terminal 107 respectively. 

Meanwhile, the switch 23 selects either the audio data fed from the MPEG AV 
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decoder 19 or the audio data input from the audio A/D converter 10 so that the 
selected signal is converted into an analog audio signal by audio A/D converter 24. 
The analog audio signal from the audio A/D converter 24 is then taken out externally, 
by way of output tenninal 108. 

Now, the digital signal that is fed and output by way of digital IN/OUT tenninal 
105 will be described below. When, for example, recording a signal input from an 
external IRD (integrated receiver decoder) by way of a digital interface such as one 
confonning to IEEE 1394, the digital signal is input to digital interface circuit 12. 

The digital interface circuit 12 performs processing operations such as format 
conversion to make the signal conform to the current system and generates a TS, 
which is then fed to multiplexer/demultiplexer 9. The multiplexer/demultiplexer 9 
analyses and generates a control signal along with other signals and converts the TS 
into a signal conforming to the current system, which is then fed to the downstream 
of the apparatus. All the subsequent processing operations are same as those 
described above for a video data and an audio data. At the same time, the 
multiplexer/demultiplexer 9 separates the components of the digital signal and supplies 
the obtained PES to the MPEG AV decoder 19 to obtain an analog video signal and 
an analog audio signal 

The signal reproducing operation of the digital interface circuit 12 will be 
described below. The operation down to the multiplexer/demultiplexer 9 is same as 
the one described above and hence will not be described any further. 
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Upon receiving the TS signal and if necessaiy, the multiplexer/demultiplexer 
9 analyses and generates a control signal and supplies it to the digital interface circuit 
12. The digital interface circuit 12 perfonns a converting operation that is opposite 
to the one it perfonns for recording to produce a digital signal confomiing to an 
external IRD, which digital signal is then output by way of digital IN/OUT tenninal 
105. At the same time, the multiplexer/demultiplexer 9 perfonns a processing 
operation for separation and supplies the PES to the MPEG AV decoder 19 to produce 
an analog video signal and an analog audio signal. While this embodiment is described 
in tenns of connection to an IRD, it can alternatively be connected to an AV apparatus 
such as TV set or a personal computer. 

While the signal recording medium of FIG. 14 is described above in tenns of 
a hard disk, it may be replaced by any appropriate signal recording medium such as an 
optical disk, a flexible disk, a magnetic recording medium such as magnetic tape, a 
semiconductor recording medium such as IC cards or some other memory for 
recording the signal coded by this em.bodim.ent. Optical disks that can be used with 
an encoder according to the invention include those adapted to record signals on a bit 
by bit basis and magneto-optical disks along with phase-change type optical disks, 
organic coloring matter type optical disks, optical disks adapted to be used with an 
ultraviolet laser beam for recording and optical disks having a multilayer recording 
film. 
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